feat(decay): native usage-aware memory decay engine + run_decay_scan MCP tool#45
feat(decay): native usage-aware memory decay engine + run_decay_scan MCP tool#45bertheto wants to merge 1 commit into
Conversation
…MCP tool
Adds a native, usage-aware decay engine so memory confidence and
obsolescence are managed by the server instead of relying on agent-side
discipline.
Schema
- memories gains access_count (int, default 0, NOT NULL) and
last_accessed_at (datetime, nullable) + index.
- Alembic migration 20260704_add_memory_usage_tracking (upgrade +
downgrade both clean).
- Dual-repo: SQLite + Postgres MemoryTable ORM extended identically;
MemoryRepository Protocol gains record_memory_access and
get_decay_candidates.
- Memory Pydantic model exposes the new fields read-only; MemoryUpdate
does NOT, so access counters stay internal and updated_at is never
distorted by read-side tracking.
Service
- record_memory_access increments access_count + sets last_accessed_at
via a dedicated UPDATE that NEVER mutates updated_at.
- Read paths query_memory (post-truncation) and get_memory call it
best-effort.
- run_decay_scan returns DecayScanResponse with per-memory
DecayCandidateAction (skip/decay/obsolete).
Decay formula (_propose_action, classmethod for unit-testability):
- effective_access = last_accessed_at ?? updated_at ?? created_at
- SKIP if importance >= 8 OR tags in {decision, architecture, critical}
- SKIP if confidence is None (no safe target without a default policy)
- SKIP if age <= 90 days
- OBSOLETE if age > 180 days AND confidence <= 0.3
- DECAY: delta = clamp(0.1 - min(access_count, 8) * 0.01, 0.02, 0.1);
proposed = max(0.3, confidence - delta); clamped at floor -> skip
MCP tool
- run_decay_scan(ctx, memory_ids, project_id, dry_run=True) in the
memory category, following the rebuild_embeddings triad pattern.
- dry_run=True previews; dry_run=False applies via update_memory /
mark_memory_obsolete (never raw SQL).
- mutates: True; tags: memory, decay, admin, lifecycle, gc.
Tests (23 new, all green)
- 16 unit + integration (test_decay_engine.py)
- 4 adapter/registry (test_decay_tool_adapters.py)
- 3 SQLite E2E (test_decay_scan_sqlite.py)
- conftest.py: InMemoryMemoryRepository stubs the new repo methods.
|
Thanks for the PR — there's clearly a lot of work in here, and memory lifecycle is a real topic I'm still thinking about how I want to handle this in Forgetful, so I'm not going to move on the I haven't landed on an approach yet though, so I'll leave this open while I think it through, and |
Summary
Adds a native, usage-aware memory decay engine so memory confidence and obsolescence are managed by the server instead of relying on agent-side discipline. Closes the gap surfaced by the 2026 open-source memory MCP landscape review: of 73 memory systems surveyed, only 2 combine decay with scheduled execution, and none track usage as a decay weight.
What changed
Schema (additive, forward-compatible)
memoriesgainsaccess_count(int, default 0, NOT NULL) andlast_accessed_at(datetime, nullable) +ix_memories_last_accessed_at.20260704_add_memory_usage_tracking(upgrade + downgrade both clean; validated on a DB copy before live application).MemoryTableORM extended identically;MemoryRepositoryProtocol gainsrecord_memory_accessandget_decay_candidates.MemoryPydantic model exposes the new fields read-only;MemoryUpdatedoes not — access counters stay internal,updated_atis never distorted by read-side tracking.Service (
app/services/memory_service.py)record_memory_accessincrementsaccess_count+ setslast_accessed_atvia a dedicated UPDATE that never mutatesupdated_at.query_memory(post-truncation) andget_memorycall it best-effort (try/except, never breaks the read).run_decay_scanreturns aDecayScanResponsewith per-memoryDecayCandidateAction(skip / decay / obsolete, delta, proposed_confidence, reason).Decay formula (
_propose_action, classmethod for pure unit-testability):effective_access = last_accessed_at ?? updated_at ?? created_atimportance >= 8OR tags in{decision, architecture, critical}confidence is None(no safe target without a default-confidence policy)<= 90 days> 180 daysANDconfidence <= 0.3delta = clamp(0.1 - min(access_count, 8) * 0.01, 0.02, 0.1);proposed = max(0.3, confidence - delta); if clamped at floor → skipMCP tool
run_decay_scan(ctx, memory_ids=None, project_id=None, dry_run=True)registered in the memory category, following the existingrebuild_embeddingstriad pattern (memory_tools.py+tool_adapters.py+tool_metadata_registry.py).dry_run=Truepreviews;dry_run=Falseapplies via existingupdate_memory/mark_memory_obsolete(never raw SQL).mutates: True; tags:memory, decay, admin, lifecycle, gc.Tests (23 new, all green)
tests/integration/test_decay_engine.py(16): pure-formula unit tests (protected tags/importance, null confidence, boundary ages 89/90/180+, usage-weight shrink, clamped floor, obsolete path) + integration viaInMemoryMemoryRepository(updated_atpreservation,get_memoryaccess tracking, dry-run no-write, live confidence decay, live obsolete).tests/integration/test_decay_tool_adapters.py(4): adapter registered, registry metadata bound, dry-run returnsDecayScanResponse.tests/e2e_sqlite/test_decay_scan_sqlite.py(3):discover_forgetful_toolslists it,execute_forgetful_tooldry-run + live round-trip.tests/integration/conftest.py:InMemoryMemoryRepositorystubs the two new repo methods.Backward compatibility
access_count = 0,last_accessed_at = NULL), so existing memories decay as if never accessed — theeffective_accessfallback toupdated_at/created_atkeeps the age signal intact.run_decay_scanis purely additive to the tool registry.Test plan
pytest tests/integration/test_decay_engine.py tests/integration/test_decay_tool_adapters.py tests/e2e_sqlite/test_decay_scan_sqlite.py→ 23 passed90dfresh-window,180dobsolete-threshold,0.3floor,0.02–0.1delta band) — these are centralized inmemory_service.pyas module constants and easy to tune.